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Abstract  We  consider  a  heteroscedastic  linear  model  in  which  the  variances 
are  a  parametric  function  of  the  mean  responses  tc-p- -rrrrrjjt  and  a  parameter  .-0 . 
We  propose  robust  estimates  for  the  regression  parameter  $  and  show  that,  as  long 
as  a  reasonable  starting  estimate  of  &  is  available,  our  estimates  of  8  are 
asymptotically  equivalent  to  the  natural  estimate  obtained  with  known  variances. 

A  particular  method  for  estimating  «f  is  proposed  and  shown  by  Monte-Carlo  to 
work  quite  well,  especially  in  power  and  exponential  models  for  the  variances. 

We  also  briefly  discuss  a  "feedback"  estimate  of  0.  < _ 
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1.  Introduction.  We  consider  the  hoteroscedustic  linear  model 


(1.1) 


Yi  =  Ti  +  °iri’ 


=  x.p,  i  =  1 ,  ....  N, 


where  {x^}  are  (1  x  p)  design  constants,  0  is  a  (p  x  1)  regression  parameter, 

{e.}  are  independent  and  identically  distributed  with  mean  zero  and  unknov/n 
symmetric  distribution  function  F,  and  are  scaling  constants  which  express 

the  possible  heteroscedasticily.  Our  primary  interest  is  estimation  of  and 
inference  about  the  unknown  regression  parameter 

Of  course,  one  could  ignore  the  {o^}  and  use  classical  methods  such  as  least 
squares  or  M-estimation  (Huber  (1977)),  but  such  estimates  arc  not  efficient. 

In  order  to  make  more  efficient  inference  about  fl,  it  is  necessary  to  get  infor¬ 
mation  about  the  {cm}.  In  one  approach  to  the  problem,  the  (c^)  are  assumed 
completely  unknown,  but  replication  is  assumed  feasible  so  that  the  {Y^.}  occur  in 
groups  of  equal  variance.  Recent  results  in  this  direction  are  due  to  Fuller  and 
Rao  (1978).  Their  results  art.*  complicated  and  the  delicate  calculations  involved 
seem  to  depend  very  heavily  on  an  assumption  of  Gaussian  errors,  i.e.,  F  =  4>, 
the  standard  normal  distribution  function.  Such  a  strong  dependence  on  the 
Gaussian  errors  is  undesirable  from  the  view  of  robustness;  the  assumption  of  an 
exact  normal  error  distribution  is  not  always  tenable  and  the  resulting  estimates 
tend  to  be  inefficient  for  distributions  with  heavier  tails  than  the  normal 
distribution.  See  Huber  (1977)  for  details  and  further  references. 

The  second  approach  to  the  estimation  problem  for  (1.1)  avoids  the  repli¬ 
cation  assumption  by  positing  a  known  form  for  the  error  variance,  i.e., 


(1.2) 


o.  «  H ( x .  ,  8,0), 


i 
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where  0  is  (r  x  1)  vector  of  unknown  coefficients  end  H  is  smooth  and  known. 

A  model  such  as  (1.2)  is  behind  the  tests  for  homoscedasticity  developed  by 
Anscembe  (1961),  Bickel  (1978)  and  Carroll  and  Ruppert  (1981).  Of  course,  in 
many  real  problems  we  suspect  a  hctorosccdastii  model  because  the  dispersion 
of  the  residuals  increases  with  the  magnitude  of  the  fitted  values.  Thus,  it 
has  become  quite  common  to  simplify  (1.2)  by  assuming  that  a ^  is  a  function  of 


x.  or  | t  . | ,  e.g. 


(1.3a) 


'l  +  h 


(1.3b) 


(1.3c) 


(1.3d) 


°i  =  °'Ti 


=  cexp(0ai ) 
=  o(l  f 


(Box  and  Hill  (1974)) 


(Bickcl  (1978)) 


(Jobson  and  fuller  (1980)) 


(Sec  Dent  and  Hildreth  (1977)  for  other  models).  Following  these  examples,  we 
will  thus  assume  that  for  some  known  H, 


(1.‘0 


a.  -•  oH*("  i ,  0*)  =  H(  i  i  ,r.  =  (r,0*)) 


Our  results  can  be  general  ize-i  to  t.iiQ  model  (1.2),  but  the  statements  of  results 
i  ' 

I 

and  assumptions  then  become  extremely  compl icaled . 

Box  and  ('ill  (1974)  and  Jobson  and  fuller  (19^0)  both  suggest  a  form  of 
generalized  weighted  least  squares.  One  obtains  estimates  of  (G»f),  constructs 
estirited  weights  and  then  performs  ordinary  weighted  least  squares.  Their 
methods  are  constructed  from  a  normal  error  assumption  and  their  efficiency  depends 
on  this  assumption.  The  maximum  lileliliood  est.itxVcs  for  0  under  the  normality 
assumption  have  a  quadratic;  ini  luor.ee  curve  and  muy  be  particularly  non-rolust. 

As  argued  above',  t h  ■  r-^cenl  1  i  teri. ♦uro  (ienuinstra los  some  acceptance  to  the  notion 
thfjt  e..  I.  for*.  should  I  o  r«»!  t  <*;.ins:  departures  f  r  c  .;;i  normality.  Out.-  purpose 


of  t  h  i  »  art  1  ■■  i  .  to  prov  i  l> 


(■  f  »  h  robu  ■.  t  .  s I  i iti.i  t  es  . 


Implicit  in  the  work  of  Box  and  Hill  (1974)  and  Jobson  and  Fuller  (1980)  is 


the  notion  that  this  problem  is  {cr.}  adaptable  (Bickel  (1980),  Wald  lectures), 
i.e.,  the  generalized  weighted  least  squires  methods  are  asymptotically  equiv 
alent  to  the  "optimal"  weighted  least  squares  estimate  one  would  define  if  the 
{o^}  were  actually  known  up  to  a  scale  factor.  Our  second  major  aim  is  to  show 
that  there  is  a  wide  class  of  (robust)  estimates  of  b  which  are  fo^ }  adaptable 
for  many  distribution  functions  f  and  mode s s  (1.4). 

2.  A  class  of  weighted  robust  estimates.  Suppose  we  have  estimates  of 
(e,  c)  which  are  N 2-consistent,  i.e., 


(2.1) 


n'Oj  -  e)  =  Op(i) 
N,;-(i'0  -  3)  =  0p(l). 


The  existence  of  such  estimates  is  discussed  in  the  next  section.  We  form  the 
estimated  a* : 

(2.2)  C'i  =  Hit)  ,  0),  ti  =  XiB0. 

Ir  the  (o.),were  known  robustness  considerations  discusseu  by  Huber  (1973, 

i  I' 

1977;  suggest  a  general  class  of  weighted  M-estimates  formed  by  solving  the 
minimization  problem  in  B 


(2.3)  Io((Yi  -  x^U/Sj)  =  minimum 

Q 

Here  ^  is  taken  to  be  a  convex  function.  If,  for  example,  o(x)  =  /2  we  get  the 

"optimal"  weighted  least  squares  estimate  with  known  weiqhts.  In  general,  the 
unknown  solution  to  (2.3)  is  denoted  3  t  . 

The  class  of  estimates  we  suggest  are  very  simply  generated  by  substituting 
i a .  j }  into  (2.3).  Taking  derivatives,  we  suggest  solving  the  equation 
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(2.4) 


Z  (x./Oj  )4'((Yi  -  XjBVo.j)  =  o, 
i=l 


with  solution  3.  Throughout  we  take  <l>  to  be  an  odd,  continuous  function.  The 
(non-robust)  generalized  weighted  ^east  squares  estimates  suggested  by  Box  and 
Hill  (1974)  and  Jobson  and  Fuller  (1980)  fall  under  the  special  case  of  (2.4) 
when  Mx)  =  x;  both  propose  possibilities  for  B0  and  t)  of  (2.1).  As  suggested 
by  the  literature,  choosing  ’!>  bounded  should  result  in  reasonably  efficient  and 
robust  estimates  of  8. 

Define  d-j  =  x-j/a,-  and  assume  that 


(2.5) 


SN  =  N*1  Z  d-d.  -*•  S  (positive  definite). 

W  1=1  1  1 


Then  by  forma  1  Taylor  series  arguments  the  optimal  robust  weighted  estimate  80 
which  minimizes  (2.4)  satisfies 


(2.6) 


N'^Bopt  -  B)  =  N'**  ?  S“1d1^(^i)/E^'(e1)  +  op(l) 

*  N(o,  E^S'^Ev')"2) . 


Our  main  result  concerning  adaptation  is  that  when  (2.1)  holds  and  hence  we 
have  a  reasonable  estimate  of  0,  then  our  estimate  S  is  asymptotically  equivalent 
to  the  (unknown)  B0pt •  formally,  we  have 

Theorem  1.  Assume  (2.1),  (2.5),  (2.9)  and  B1  -  B8  below.  Then 


(2.7) 


N 2 ( £  ■  Snnt )  ^  o , 


so  that 


(2.8) 


fF(3  -  e)r*  N(o,  E-iV^E**)-2) 
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That  j3  is  robust  against  outliers  in  the  errors  when  >p  is  bounded  can  be 
seen  by  combining  (2.6)  -  (2.7).  The  resulting  influence  curve  is  strikingly 
similar  to  the  unweighted  case  in  homoscedastic  models. 

In  stating  assumptions  and  proofs  we  simplify  (1.4)  to 

(2.9)  o.  =  exp( h(t .  )d  ) , 

where  h  is  a  function  from  R  to  Rr.  The  model  (2.9)  includes  models  (1.3a)  - 
(1.3c),  but  it  is  not  strictly  necessary  for  the  validity  of  our  results.  Our 
reason  for  considering  only  (2.9)  in  the  formal  aspects  of  this  section  is  to  avoid 
making  already  cumbersome  notation  needlessly  complicated.  Generalizations  to 
the  model  (1.4)  require  that  H(-,-)  be  smooth. 


Bl. 

B2 . 


B3 . 


B4. 


B5. 


Here  are  the  assumptions. 

2 

ip  odd,  F  symmetric,  (>  ,  )•  ",  E’i<  *>o 

As  r  -*>  o,  s  -*•  o, 

€v( (Cj  +  r)(l  +  s)) 

=  rEt!> *  +  o( '  r ;  +  Js!), 

There  exists  C  such  that  for  all  5<1,  as  K  » 

F  supiv’( (>■  ,  +  r ) ( 1  +  s))  -  +  r')(  1  +  s'))|: 

>  1  1  \ 

:\r[  ,  •  r '  ’  ,  ls|,  is'^K,  [r  -  r'\  and  )s  -  s'k»K£j 


As  r ,  s  *  o , 

E ( V f ( •  j  ♦  r)(l  +  s))  -  v(.  j))2  -  o 


I’m  supfi  Jx,;  ■  +  j  Jh(T.)’  ]  )N~'i  =  o 
N -  iuq  1 
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B6.  sup(N_1  T.  ( i  i  xi  |  | 2  +  |  I  h  ( x  - )  [  !2))<« 

~  N  i  =1 

87.  The  functions  =  H(r-,0)  are  bounded  away  from  zero. 

B8.  On  an  interval  I  containing  all  the  { t ^ } ,  h  is  lipschitz 

continuous.  I  may  be  infinite. 

The  proof  is  given  in  Appendix  B.  Conditions  EU  -  BJ5  are  similar  to  those 
used  by  Bickel  (1975)  in  his  study  of  one-step  M-estimates  in  the  homoscedastic 
model.  Condition  J57  merely  insures  that  we  do  not  have  infinite  weights  and 
condition  B8  assures  us  that  when  a.  =  H(t1-,8)  =  exp(h(x^)0),  the  function  H  is 
sufficiently  smooth.  Details  verifying  that  (2.8)  actually  follows  from  (2.6)  - 
(2.7)  and  the  assumptions  of  Theorem  1  are  easy  to  fill  in. 

3.  Estimation  of  0.  In  the  previous  section  we  have  shown  that,  except  for 
certain  technical  conditions,  one  can  construct  robust  weighted  estimates  of  8  as 
long  as  one  has  available  estimates  of  6  and  6  which  satisfy  (2.1).  Preliminary 
estimates  80  satisfying  (2.1)  are  readily  available  and  include  (under  reasonable 

assumptions)  ordinary  least  squares  estimates  and  ordinary  M-estimates;  details 

,1 

of  sufficient  conditions  for  this  are  available  from  the  authors  in  the  form  of  an 
earlier  version  of  this  paper.  Bounded  influence  regression  estimates  could  also 
be  used;  see  e.g.,  Krasker  and  Welsch  (1981).  In  this  section  we  propose  a  class 
of  estimates  of  0  which  are  robust  and  satisfy  (2.1).  There  are  of  course  many 
possible  ways  to  construct  such  estimates,  but  our  method  has  the  necessary 
theoretical  properties  as  well  as  encouraging  small  sample  properties;  see  the 
next  section  for  details. 

To  motivate  our  estimates,  suppose  the  were  known,  the  'c.;  satisfy 

(1.4)  and  the  density  of  f  is  proportional  to  exp(-p(x)),  where  o  and  o'  =  are 
as  in  the  previous  section.  This  device  is  common  in  robustness  studies;  see 
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Huber  (  1  977  ),  Bickel  and  Doksuin  (1931)  and  varrol 1  (1  980)  for  examples.  In  this 
instance  the  log-likelihood  for  0  is  (up  to  a  constant) 


(3.1) 


N  f| 

1(9)  =  -  j:  logll(  i  i  .;, )  -  r.  r((Yi  -  Ti )/H(ri  ,0) ) 
i-1  i-1 


Taking  derivatives  in  0  suggests  that  we  solve 


(3.2) 


0  ^ 


if») 


1  =1 


/(Y.-  t .  )'i/v .  -  t .  \  | 


V"  , 


Because  the  term  in  brackets  in  (3.2)  is  not  bounded  and  hence  would  in  general 
lead  to  an  unbounded  influence-  function  for  the  estimated  0  and  an  overall  lack 
of  robustness  in  our  estimation  procedure,  we  follow  t he  common  device  used  in 
the  homjscodastic  case  by  Huber  ( 1 9^ 7 )  and  Bickel  and  Do k sun  (1931)  of  replacing 
xi)'(a)  -  1  by  a  function  x(‘)>  as  well  as  replacing  i-j  by  t-j  r-  i'0 ,  thus  leading 
to  estimates  obtained  by  solving 


(3.3) 


N 


0  =  CN(ft)  «  >:  V  (V.  -  t^/Cftj.u)  •^!(t1>9)/n(ti,0). 


Probably  the  most  co iv  on  choice  of  \(  )  in  the  homosccdast  it:  rase  is  that  for 
the  classical  Proposal  ?.: 

x(y)  -  -?(y)  -  /v2(x)(^)‘^exp(-\  /?}dx. 

This  choice  of  y(.)  cives  f 'su' V.i  i ». *'l  ur»  ce  to  our  estimates  cf  9,  and  thus  ini  jht 
reaoona!  ly  be  preferred  in  our  problem  to  y  >(y)  -  1,  just  as  it  is  in  the  homo- 
scedastl'  case;  soc  also  Mu!.'*  (];77,  p.  J>)  for  t.rtain  opt  i :  lit  y  properties 
of  this  choice.  In  tho  race  tv*  1  h"  -pori.il  ;r,0vle1  (^.9),  wo  have 

f}[ /  6 )  •  ^  j  *■ 7  ( 1  i  ■  f  i )  ^ *  1  j  h  ( t  j  ;  . 


(3.4) 
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We  make  the  assumptions  that  •  (.)  is  an  even  function  with  'A(o)< o,  x('”)'jo.  In 
the  model  (1.4),  -r  is  a  free  parameter  defined  so  that 


(3.5) 


r  v  ( (  y 


l 


■  )/'  )  --  o. 
1  I 


In  the  model  (1.3a)  we  have 


0-1  o  Ci' 


=  /  1  N. 

ki  1 


In  many  models  (such  as  (1.3a),  ll.'<b),  and  (1.3c)  with  ;  ■  o )  one 
can  show  that  solutions  to  the  equation  0,^(0)  -  6fJ ( o ,5  + )  =  o  exist.  But  we  have 
been  unable  to  show  that  the  solutions  are  unique,  altnough  in  all  of  our  examples 
unique  sol  itio^s  have  been  obtained.  More  importantly,  one  may  not  wish  to 
consider  all  poss i ble  values  of’;,  e.g.,  in  model s  (1.3a)  -  (1.3c),  one  may 
reasonably  wish  to  restrict  );>  ;  —  1 . 5 .  For  these  reasons,  we  suggest  the  following 
procedure : 

(3.6)  minimize  f  | G^, (•"')!  J  =  ;  j  Gf,(r ,r  *)  |  |  on  the  interval 

•1  J.  If  the  solution  is  not  unique,  c'oose  that  one 
wi th  sinal  lest  J  j  i  . 

] 

The  solution  to  (3.6)  is  thus  well-defined.  In  all  of  our  examples  when  0  is 
unrestricted,  the  solutions  to  (3.3)  and  (3.6)  have  coincided.  In  tne  examples 
in  which  we  have  restricted  ,  (3.6)  has  always  had  a  unique  solution  even  when 
(3.31  has  not  had  a  solution  in  the  restricted  space. 

An  appealing  feature  of  there  estimates  is  that,  they  are  natural  general¬ 
izations  of  the  classical  Huber  Proposal  .?  for  the  homoscedastic  case. 

Theorem  2.  Assume  (2. 5), (2. 9),  (3.5),  B_5  -  B8.  Further  assume  that 
N2(B0  -  6)  =  Op(l).  Then  under  C_1  -  C5  below,  if  0  solves  (3.6),  then 

(3.7)  *  -  •  =  opcr'"2). 


(3.7) 


u 


Here  are  the  conditions: 

O 

Cl  c"E;.l  (:  j ■  ^  ,  and  is  nondecreasing  on  [o,  °) . 

C2  As  r,  s  k  ^  for  A(  .  )'*0 , 

E;<  ( ( c  i  :  r)(l  +  $))  =  A(  Js  +  o  ( I  r  1  +  [s! ). 

C3  Condition  B3  holes  for 


C4  Condition  B_4  holds  for  <  • 


C5  If  ^  is  the  minimal  eigenvalue  of 


then  lim  inf 


The  proof  is  given  in  Appendix  C.  The  conditions  are  hardly  onerous ,  being 
similar  to  those  of  Bickel  (1.75),  and  with  only  C5  affected  by  the  hetero- 
scedastic i t y .  Further  details  of  implementation  are  discussed  in  the  next  section. 

One  can  also  introduce  redescending  M-estimates  by  using  y  redescending  to 
zero.  No  change  is  needed  Theorem  3,  while  an  estimate  for  0  can  be  obtained 
by  doing  one  or  two  steps  of  Newton-Raphson  for  (3.3)  from  any  estimate  satisfying 
(2.1)  Proofs  are  similar  to  those  given  in  the  appendicies. 


4.  A  Monte-Carlo  study.  Because  Theorem  1  is  an  asymptotic  result,  we  per¬ 
formed  a  small  Monte-Carlo  stud,  to  assess  the  small  sample  properties  of  6.  The 
model  was  simple  linear  regression 


(4.1) 


+  “lCi  + 


i  +  Jri’  1  =  l'  ••••"• 


In  the  study,  the  -.Ci  -  were  equally  spaced  between  -2  and  +2,  and  we  chose  to 
study  the  model  (1.3a) 
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The  experiments  were  each  repeated  two  hundred  times  under  the  following  circum¬ 
stances  : 

(a)  N  =  2i,  {e.j}  are  standard  normal,  a  s  .25,  s  2,  Bj  *  1 

(b)  N  =  41,  {e.}  are  N(o,l)  with  probability  p  =  .90  and  N(o,  Var  -  9)  with 

p  =  .10,  a  =  .25,  3Q  =  4,  =  2.  The  { e ^  1  are  said  to  have  a  contam¬ 

inated  normal  distribution.  Such  distributions  are  often  used  in  ro¬ 
bustness  studies. 

We  made  two  choices  for  b.  First  was  vp ( x )  =  x,  which  yields  the  usual 
weighted  least  squares  estimate  and  the  second  was  Huber's  \b(x)  =  max 

A 

(-  2.0,  min(x,  2.0)).  This  gives  a  version  Bp  of  our  robust  weighted  estimates. 
In  constructing  we  defined 


(4.2)  v  ( y )  -  ;«2(y)  -  ftp?  (y  )q  (y  )dy , 


where  9 ( • )  is  the  standard  normal  density  function.  This  particular  v(- )  is  in 
standard  use  for  Huber's  Proposal  2  in  homoscedastic  models.  The  choice 
x(y)  -  yi'(y)  -  1  was  not  chosen  because  of  unbounded  influence  functions  leading 
to  robustness  difficulties. 

X  A 

Both  3^  and  Bp  were  constructed  as  follows: 

Step  (i):  L£t  B*  be  the  unweighted  Huber  Proposal  2  estimate  (9  =  o)  with 

.  i 

X  given  by  (4.2)  and  ;(x)  =  max{-2,  m i n ( x ,  2)). 

Step  (ii):  Solve  (3.5)  for  (c*,  6  +  )  form  inverse  "weights"  w?  =  (1  +  |t^|)^*. 


Step  ( i  i  i ) : 

(4.3) 


Solve  a  weighted  Huber  Proposal  2  by  simultaneously  solving  (2.4) 
for  the  desired  function  i/-  and  the  part  of  (3.5)  given  by 
2  a  ( ( Y  •  -  x  •  '•  )  / aw . )  =  o. 


The  result  is  R 

Step  (iv):  Repeat  steps  (ii)  and  (iii)  to  obtain  t^  =  x^8Q,  0  ,  n,  B. 
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I  j 

The  algorithm  given  here  was  chosen  so  as  to  reproduce  Huber's  Proposal  2 
in  the  hoinoscedns t ic  case  f)  -  o.  Direct  application  of  the  results  of  Section  2 
involves  only  solving  (2.4)  in  Step  (iii)  jnd  gave  results  essentially  indis-  ; 

tinguisable  from  those  reported  h*re.  In  solving  for  (■; ,  a)  we  used  the  sub-  \ 

routine  ZXGSh  of  the  IMSL  library.  : 

In  Table  t 1  we  list  part  of  the  results  of  the  study.  The  values  listed  are  < 

ratios  of  mean  square  errors  for  estimating  in  model  (4.1),  the  ratio  being  . 

with  respect  to  the  "optimal"  robust  method  one  would  use  if  w.j  =  (1  +  jx-jl)®  ;  : 

were  actual1;/  known,  i.e.,  solve  (2.4)  and  (4.3)  simul tanucusly  with  the  known 
weights.  The  study  is  fairly  small  but  it  does  seem  to  indicate  that  our  robust 
weighted  estimate  will  work  in  situations  in  which  heteroscedasticity  is  suspected. 

It  is  important  to  note  that  our  estimate  has  MSE  never  more  man  10%  larger 
than  the  unknown  estimate  formed  with  the  correct  weights,  and  seems  to  do  better  t 

than  unweighted  estimates  when  f  o.  Note  also  the  robustness  feature;  the 
efficiency  of  the  weighted  least  squares  estimates  (even  the  "optimal"  one)  depends 
heavily  on  the  normality  assumption  and  is  not  very  high  in  the  contaminated  case. 

All  of  these  results  tend  to  support  the  appl icabil i ty  of  Theorem  1.  1 

We  repeated  the  experiment  but  with  the  model 

- i  =  ~exp(e ; t j ! ) . 


The  MSE  results  are  reported  in  Table  2.  These  results  seem  to  indicate  that  our 
theory  is  applicable  for  a  variety  of  models  for  the  (r. K 

Statistical  inference  for  °  is  also  possible.  We  use  the  following  general¬ 
ization  of  methods  suggested  by  Huber  (1973)  for  the  homoscedastic  case.  Using 

!  .  A 

(2.8)  of  Theorem  1,  we  estimate  the  covariance  of  N*(6  -  6)  by 


(4.4) 


A  A  /s 
K(  E-i  ^  )S_1  ( E-:  ' ) 
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where 

A 

'  “  f .  *  -  N'  l‘r((Yi  -  xitf)/ci) 

K  -  1  +  (P  +  ?)(1  •  )/(N) 

/\  f-l  .  ,  -2 

"J  ”  ^  •  xixr*i 

/\  ^\ 

d > ■  ti  r  ‘  ;  >  similarly  to  U  * .  In  our  Monte-Carlo  experiment  we  constructed 

:on*’3t»i  j  ■. n'  *  <als  for  tfie  lope  parameter  tfj.  in  (4.2 },  using  (4.4)  and  t- 
:5rr°rta  prints  witH  *'  -p-r-N-4  degrees  of  freedom.  The  intended  coverage 
pro^ac*' i*v  wa  ,  95'*;  in  non,  ot  tn*  cases  did  the  achieved  coverage  probability 
fail  be)  \t  9?  ,  .v>d  in  the  majority  of  the  cases  it  was  at  least  9^  . 

W-.  a '  so  attempted  to  sol'-e  equations  (2.4)  and  (3.4)  simultaneously  using 
tne  IMbL  routine  ZSYSTM.  Our  experience  was  much  like  that  of  Frochlich  (1973) 
in  that  the  algorithm  converged  most  of  the  time  but  not  always.  Oent  and  Hildreth 
(1977)  were  able  to  show  that  the  difficulties  experienced  by  Froehlich  could  be 
overcome  by  sophi stica ted  optimization  techniques.  We  suspect  that,  the  same  holds 
for  our  problem. 

Tne  particular  method  for  estimating  0  *(•■-,  o  )  outlined  in  Section  3  and 
explored  in  this'section  is  recommended  for  models  such  as  (1.3a)  -  (1.3c)  which 
satisfy  (2.9).  In  model  (1.3d)  an  alternative  procedure  is  preferable,  because 
we  car.  exploit:  the  relationship 

ji2  =  ‘i f  v.2- 

Here  one  would  obtain  initial  estimates  of  (u.  ,  •'t„5  by  (robust)  regression  tech¬ 
niques,  as  along  the  lines  or  Jobson  and  Fuller  (1980),  working  with  the  squares 
of  the  residuals  from  a  preliminary  fit.  One  would  then  do  one-step  of  a  Newton- 
Raphson  towards  solving  versions  of  (3.4)  which  are  obtained  by  working  with 
U,  *2 )  and  following  the  line  of  reasoning  in  (3.1)  -  (3.4).  Monte-Carlo  work 
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which  will  be  reported  elsewhere  indicates  that  this  technique  can  be  quite 
successful . 

5.  Feedback .  In  the  case  of  normal  errors,  Jobson  and  Fuller  have  suggested 
using  the  information  about  S  in  the  terms  a-  =  H(t  - ,v  ).  This  essentially  reduces 
to  maximizing  (3.1)  jointly  in  (*),  $) .  In  a  very  nice  result  they  show  that  if 
the  error  distribution  i^  exactly  normal  and  if  (5.1)  is  exactly  correct,  then 
improvement  over  the  weighted  least  squares  estimate  can  be  achieved.  It  is  clear 
that  such  feedback  procedures  will  be  adversely  affected  by  outliers  or  non-normal 
error  distributions,  and  it  is  not  clear'  how  to  robustly  modify  them. 

In  cases  where  using  feedback  is  contemplated,  a  second  form  of  robustness 
must  also  be  considered,  i.e.,  robustness  against  misspecif ication  of  the  function 
H  in  (3.1).  Carroll  (1981,  unpublished)  has  shown  that  as  long  as  H  is  correctly 
specified  to  order  Q(N  J) ,  the  asymptotic  properties  of  the  weighted  estimates 
((2.4),  (3.5))  are  the  same  as  if  H  were  correctly  specified;  in  this  sense  our 
weighted  estimates  are  robust,  against  small  errors  in  specifying  H.  Carroll  also 
chows  that  such  robustness  is  not  the  case  for  feedback  estimates.  In  fact,  any 
gain  *rom  feedback  can  be  more  than  offset  by  slight  errors  in  specifying  H. 

Since  our  primary  interest  is  in  and  c-j  =  H(s1-,  6 )  is  at  best  an  approximation, 
we  suggest  that  feedback  should  not  be  automatically  preferred  in  practical  use. 

6.  An  example.  In  figure  *1  we  plot  the  outcomes  of  113  observations  of 
Total  Esterase  ;'Ci)  and  Radio  immunoassay  -  R I A  Y  -  ; ,  made  available  to  us  by  Drs. 

0.  Horowitz  and  0.  ProUd  of  the  National  Heart,  Lung  and  Blood  Institute.  The 
data  are  clearly  heteroscedast ic ,  so  we  tit  the  model  (4.1)  with  variance  model 
(1.3a),  estimation  done  as  in  the  previous  section.  The  results  are  summarized 
in  Table  *3.  Since  6  appears  to  be  fairly  large,  the  results  of  the  Monte-Carlo 
indicate  that  weighting  should  bo  of  real  benefit.  The  confidence  limits  on  9 


1C 


were  obtained  by  bootstrapping  (using  60  simulations).  In  the  weighted  cases 
the  standard  errors  for  6Q  and  8j  were  obtained  from  (4.4),  as  well  as  by  the 
bootstrap.  The  weighted  results  are  fairly  close  together.  While  our  purpose 
in  presenting  the  numbers  is  merely  illustrative,  we  note  that  the  values  of 
0  suggest  that  a  logarithmic  or  square  root  transformation  might  stabilize  the 
variances  (see  Box  and  Hill  (1974)).  We  also  fit  a  quadratic  model  to  the  data 
with  1 ittl e  change. 
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Ratio  of  Monte-Carlo  Mil:  for  the  different  estimators  to  the  Monte-Carlo 

MSE  of  the  "optimal"  robust  estimate  with  known  weights.  Ihese  MSPs  are  for 

estimating  the  slope  m  the  model  Y.  =  3,  +  B.c.  +  o.c.,  where  o.  satisfies 
'■  1  a  0  1  l  i  t  i 

(1.3a) . 
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Results  of  the  analysis  on  the  data  for  Figure  #1, 
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AITI  NT'IX  A 


S  A  A 

ine  following  general  theorem  will  be  used  when  studying  bp,  0  and  6. 


Theorem  A.l.  Let  g^ ,  k.  ,  and  A( :  ,  il  (=  g^t,  k.^.,  and  A(o,i,\0)  be  sequences 
of  positive  constants  such  that 


(A  I)  limfsup  k.  +  k.g.  +  \  "  A =  0 
X-«*  i<N  1  1  1 


1  7  7  ?  .  ! 

(A. 2)  sunf  ^  k“  ■*  ktgT  +  X  •  g.k.  )  =  C, 


X  i=l 


i  r  l 


i  v'l 


Let  be  a  function  from  R'^  to  R^  satisfying 


(A. 3)  Hi .  fc.  ,0,0)  =  0  for  all  i, 


A.  4)  1  im  sup  }•{ sup |<J> •  (t . ,  r , s  )  -  y .  ( ,  r ' ,  s * ) ! :  ; r  \ ,  |r'  j ,  js  ! ,  is '  1  k  and 

k  *0  11  L  1 


:r  -  r’ j , 'S  -  s’ 


k  }  s  ‘"o  ‘Si 


for  some  C.  and  all  0  •  ,•  •_  ]  and  all  i , 


(A. 5)  sun  g  k  i'(i.  fc.  ,r,sl  - 
i<X  1  1  1 


i^UpO.O)  -  A(c,  Or'1  =  ofjri  +  |s|j  , 


1  im  sup  g'r  i:c.  (r  ...r.s)  -  <J>.  (t. ,  ,0,fl))2  =  () 
r , s -0  i-.X  1  1  1  3  1 


■) 

and  lim  sup  g"“  J!:d(t . ,(),;))  •: 
X~*  i-x  1  1  1 


Lot  u!l),  a!21,  and  a(3) 
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be  functions  from 


(A. 12)  l\.(A)  '  Uv(0)  -  rru  fA)  -  n  (0))  =  0  (1 ) 

for  each  fix  \l  A,  ami  that  there  exist?  K  depend i no  upon  M  but  not  6  such  that, 
for  all  0  <  6  •'  1  and  all  N, 

(A.  13)  E  sup{||U  .(A)  -  b,,iA*)  |  | :  |  |.s|  |  <  M,  jy*j|  <  M,  1 1 A  -  A  *  J  J  <  M<5)  <  K5 


Since  for  any  >5,  we  can  cover  a  hall  of  radius  M  in  UM  with  a  finite  number  of 

balls  of  radius  •$,  (A.  11),  ( A •  12")  and  (A. 13)  prove  that,  for  each  0  <  6  <  1, 

livii  pf  sup  |  JlLCAl  -  IJV(0)  -  N  '*•  T  A('!  ,i)cd  J  (A)  z  .  jj  <  K6)  =  1  , 

r>"  liAlIsM  N-  A‘  A  1  ^ 


which  proves  the  theorem. 

To  prove  (A. 11),  nolo  that  by  (A. 3), 


N  ‘  T  f (U  (A)  -  Um.(0)) 


=  N';i  y  !•(<(.  fr.  ,«}J;(A)  ,  a>-!(:  VI  -  <Me-,0,cme  •  +  a(.5)(X 

^  i  j  ’  i  ’i  -  u.  i’  ’  "  -i  —  1  — 


,'c  n-  xt  have  by  (A.l),  (A. 7)  and  (A. 8)  that  for  all  large  N, 


(A.  14)  |  !.\  .  +  Y3)(' .) 


and  also  by  (A. 7)  an  I  (A. ft) 


‘•'Y  i  •«!  h;;» 


>  •  vi('.'i,v,0))  --  Ap:  ,i)f.xY  (A)  +  oCg.kJ  !a!  i) 


uni  frr::il v  in  i.  There fo 


X  X 


KilUvU)  -  =  N'  v  (A)-  •  +  0(N';  y  g.k.  ||z.| 

\  -  -  j  =  1  x  -  -  1  i^l  1  1  -1 


+  K  '  l  A(>,i),!lj(0)a!2J  (_:.)) 

•  i  1  X 


By  f'A.l),  tA.7)  and  (A. 8),  the  last  term  on  the  RHS  is  o(lj.  By  (A. 2), 
(A. 9.)  and  the  Cauchy- Sehuar::  inequality,  the  second  tenn  is  boi:nded  by 


\r 


of  i  -yj  -  o(ii 

i=l  1  1 


so  that  (A. 1.1)  holds.  Then,  again  using  !A.14),  wo  have  that  for  X  large, 


h  VarfUN(A)  -  l\.  f  0 .) ) 


N 


£  (2N  1  l  gr  !  ir  -  i!“)?»ip  gy  ,  a!  ^  (A)  »«!“■*  CA)) 

i=l  iiN  1 


l  - 


-  ,‘.(^,0,0))-  *  F.  j  |\:" 


.  X 


i =1 


ri,o»o;11(i3'1  u)i ; 2  . 


The  second  term  pn  the  Ri IS  is  o(l)  by  ( A. 6) - (A. S) .  It  also  follows  from 
\/ 

(A. 6) -(A. 8)  that 


sup  g.“  E .  ,  a!  !/.)  ,  (1))  -  ;  l.v. .  ,0,0)  =  o(l)  . 

i<N 


1  — ' 


i  i 


Therefore,  y\.12)  is  proved  by  applying  (A.  2)  and  (A.  9),  Finally, 
the  RHS  of  (A.  13)  is  less  than  or  equal  to 


by  (A. 14) 


. 


N 

?M  * 


x  iixjik , o.;i)(a>,c.‘2)(«) 

'  "1 

-  <,.(£.  .nP^A*)  ,up(_A*)|:  IN!  ^  M,  ||A*||  <  M, 


i=x 


11  1  —  i 

| | A  -  A* | |  <  M5 } 

...  N 


+  N  ‘  l  Y  sup{<&.  (f.  ,  rxp-'(A*)  ,  ctp^  (AA)  )  (ap  (A)  -  ft^( A*)) 
i--J  11  1  1  1  “ 


Ml  < 


M,  I i A * i I  £  M,  I | A  -  A* | |  <  M5}  . 


By  (A. 2),  (A. 8),  (A. 9)  and  the  Cauchy -Schwarz  incqualit) ,  the  first  term  does 
not  exceed 


5U,,(  I  („!*) )(■„■'  -  Vj*  . 

1-i 


By  (A. 2),  (A. 4)  and  (A. 9),  the  second  term  docs  not  exceed 


N 


6M2  Cq  N"**  gjkjd  +  |  Uj  !  !)  *  o(l)  . 


i=l 


There fore ,  (A.  13')'  i s  vcri  Tied . 


□ 
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Proof  of  Tnio;i.'i;i  1.  For  A  j  and  A  ^  in  R‘\  A,  in  R‘  ,  and  A  =  (A  j ,  A  2  > A  3)  » 


define 


0  )(,S 

ai 


"  N'  --  i  ‘ :  1 


h;(A)  - 


1  :  ;> 


«S’\A) 


cxp(-!u  ;7,r  \’  J)  +  (Ik  (0) 


-  h.  (A.  1)0  -  1 

j  —  • 


a  ' ’  '  C  A )  =  d  a!  '  -1  (/ 

]  11- 


Def *; : ■  e  the  mcej 


ILi/  J  “  N’*"  J1  iMU  -  -  a1 

]\  -  .  ^  1  v  i  1 


Note  that  (2.  *5)  /an  be  rewritten  a' 


(B.l)  nNF'-'J(r-d)  ,  .\V-‘  f  •  N  “  0 


!l)(A)j(i  ♦  +  ap'(Al)  . 


J  -  "  "  1  1  - 


Letting  C.  =  1,  k.  =  N~  j(i  +  lid.  I!  +  IlMfj)!!).  c.f-^r.s)  -  *((c.  -  r)  (1  +  s)), 
df  •=  z.,  and  A(^)  -  r  }:v ’ ,  the  condit  ions  o*'  Theorem  A.l  are  implied  by 

(2.5)  and  hi  -  bd,  so  for  all  M  ^  0, 

(B.2)  sup  i  | U., (A)  -  -  A( ISA  ,11  =  o  il]  . 

|  |  A  j  j  <  M  N  -  '  _1  1 

Not'  by  Uieb/^hev*  -  theorem,  HI  and  K5, 


I 


V-°J  -  y»  ■ 


In  proving  the  Theorem,  we  will  not  assume  that  fS  actually  solves  (2.4),  but 
rather  that  t.he  l.h.s.  of  (2.4)  evaluated  at  £  is  less  than  twice  it's  infimum 
over  all  £.  However,  as  seen  in  Huber  (1977),  (2.4)  will  have  a  unique  solution 
if  is  monotone.  From  the  last  equation,  we  have 

A*  =  -(AftOSf1  UN(0)  =  Op(l)  , 


so  that  by  (B.2),  IJ(A*)  -  op(l).  Consequently,  by  (B.l), 


(R.3)  l!,.(K!i(B-0)  ,  N?:’(G-0)  --  o  (l) 


By  (2.1),  (0-0)  «-• 


0  (N  ‘),  so  we  need  onlv  establish  that 
P 


(B.4)  CP- PO  =  0  (N‘?0 


to  conclude  from  (B.l)  and  (B.2)  that  (2.7)  holds.  But  by  (B.3),  (B.4)  holds 
if  for  each  n  >  O^.'c  >  0  ;uid  M, ,  there  exists  M0  satisfying 


(B. 5)  )'(  inf  inf  inf  Muv(A)|!  >  n)  >  1  -  c  . 

1  !  A  !  1 1  !=  Mz  ||A,||::M1  I  !a3I  |  £  M,  *■- 


Now  (B.S)  f.  Hows  from  (B.2)  in  a  manner  quite  similar  to  Juivckovn's  (1977) 
proof  of  her  I  cimmi  5.2. 


APPENDIX  C 


Proof  of  Theorem  2.  I'or  A  ^  in  R^,  A,  and  4  =  (4p42^*  define 


hi (A)  = 

h(ri  + 

xi-l  »"*> 

y 

(C.l) 

:^(A)  = 

e.\]){  -h 

i  ( A )  A  ^  N‘?i 

+  <hi 

(0)  -  h 

.(A))9)  -  1  , 

(0.2) 

v!2)(A)  = 

N'"1*  d. 

A^  (sec  (2. 

.5))  , 

> 

and 

(0.5) 

a[V,(A)  = 

IwlO) 

-  • 

Ilien  let 

$  f  i  Y  y  -  )  “ 

A  ( ( x  • 

-  0(1  -  y)) 

and 

define 

the  process 

U'fA)  =  -N'*4  V  <|  (A)  .apV^lOifT.)  +  af3)  (A))  . 

.N  -  i=  L  11  1  11 

Note  that  (5.6)  can  be  written  as 

|  !lv'  ^(Sp- 2j  ,  N  ~  ( 9  -  6 ) )  [  [  =  minimum  . 

However,  by  (5.5),  Cl  and  Chcbyshev's  inequality, 

(C.4)  WN(0)  =  0(1) 

so  that 

B0- e)  ,  N  “(0-'}) )  =  0  (1)  . 

lie  can  therefore  prove  (5.7)  by  showing  that  for  each  >  0,  r  >0  and  Q  >  0, 


there  exists  M-  >  0  such  that 


(C.  5) 


*  ‘V  !|A2! 


>  M,.1  >  Q1  s  1  -  e  . 


PI  inf  { |  | UN (£..)'  i : 


'  -- 1 


We  will  prove  (C.5)  by  modi  tying  the  proof  of  Jureckovd ' s  (1977)  Lemma  5. 

first  apply  Theorem  \. 1  with  =  h.(0),  g.  i  1,  A(?,i)  =  A(X) ,  and 

k.  =  S  Jihfr.)  *  !  j  x  -  1 1  +  1 1  d .  ! .  ’ll  sen 
i  i  1  i  1  i  ' 

...  N  m 

sup  I  |WV(A)  -  WUOJ  -  AfXJN  -1  l  h(..)a  U(A)j|  «  o  (lj  . 

|  i  A  |  |  <  M  X_  N"  i=l  1  1  “  p 


By  a  Taylor  series  expansion, 


h(ci)A2 


0^(0)  -  hjU1)-  +  o  (.\r?s)  . 


Thus,  by  C5  setting 


GvfA'j  =  N"*4  7  h(t.  )T  (h.(0)  -  h.(A)V7  , 

i=l  1  1  1  ~ 


we  obtain 

(C.6)  sup,  |  iW’(A)  -  Wv(0)  -  AtX)HA,  +  Cv(A)i!  =  o  (lj  . 

!|a|(Hm  N“  N  x  _  p 

Now  fix  e  >  0,  >  0,  Q  >  0.  [fsc  (l.31  to  choose  v  such  that 


PCS hvN(0) | i  >  y/2)  <  ;/:  . 


Define 


1) 


sup  sun  |U\.(A)'|  . 

n  i  ia  1n^Ml 


Then  D  <  ^  (G  depends  only  on  A,)*  Define  M,  by  ( I X )  M,/2  -  v  -  D] 

—  i  i.  'X)  4. 


L 


.  We 


=  Q. 


? 


Using  C 5  and  (C.6),  find  such  that  >  >.  /2  and 

P(  ,  sup  |!W„(A)  -  K'  (0)  -  A(X)HA  ?  -  GL.(Aj  1 1  <  y/2} 

!  |  A  2 1  ]  =  M  and  |  I A  3  I  !  <  N  N  2  “N  - 

>  1  -  r/2  (N  >  Nq)  . 

If  |'|A7||  =  M, ,  !  I A 1 1  I  "  aiK"  ^  -  ^q»  then  with  probability  at  least 

1  -  e, 

A,Wn(A)  2  -M2!  |Wn(0)||  +  £  HA,  A(X)  -  M,D  -  M7y/2 
*  [A(X)X^M2/2  -  y  -  DJNL,  =  qm2  . 

Since  X  is  nondecreasing  on  [0,"-0  by  Cl,  A  2  IV^,(A  A  7  s)  is  a  nondecreasing 
function  of  s.  Thus,  ;|A2||  2  M7  implies 


A2Wn(A)  >  A2  V-l »  M2-2*  I— 2  ^  l_1) 

>  (|  JA  2 1  I/M,)  (M,A  2  j  |A  2  |  |  1  KjjCAj  ,M2A2||A2ir1)) 

,  *  !  I A  ->  1  IQ  • 

l 

Thus , 


•1  ,W v(A) 

Pf  inf 

iMsMi  ‘ 

I  U2|  |*m2 


Q)  H  ■  e  , 


□ 


which  with  the  Cauchy -Schwars  inequality  proves  (3.2). 


t  turn  7  .  *  i  /  v  i  ►  r  *T!-.* 


tf/VCC4SS//r/'t-J> 


DOCUMf  a  i  a  i  u i‘t  r  AfiF; 


AI’CSRATK— o  I  -0219 


4  *iyli  ''.m.i  r.  iJWTFrj 

,.y  .  .... _ 

— Pobusl  estimation  in  Hetcrosrerln-*  Linear 
Model s  9 

7,  >»J  T  moWmj  “  *  ‘  ’  ^  ~ 

Raymond  J^arv  oll  <mm1  David^upport/ 

P f  i;  r  '  ■  ii *.«  "i  O  ■'  *•  . *’■  •<  i  7  \  r  n  am  ‘  •  i*  t  ■■  ,  >•  i 

University  of  North  Carol  i  na /Grape l  Hill 
Department  of  Statistics 
Chapel  Hill,  N.  C.  27514 

II  COi.  ‘  «;>L  !»tf  '  *  i  ,-|  I.  AM  '  A*i  ;  I  ai  ».»»•»  . 

Air  Force  Office  of  Scientific  Uesearch 
Colling  Air  force  Base 
Washington,  DC  2077? 


1  4  W  'll  ’  r  Hi  n  A  .  •  t.  ,  I  i  \  v  t  A  A  ,  • 


\  ' 1  *.\  ■  I  f  '  .  \  '■ 

HI  *  i  Ml  -1  .i  ;\.  ,  }■<  iHV, 


j  Interim  jupt  /  / 

|  '■  |  r  irniili  * 

'  '/Vc*?  '  ■■*>■***■  A~ltl 

(J kK fosn-ce-oosoy £ 

K~~fvAJ<oF-Mc£ 

•  .*  f.HHM  ui  I  i>  J  ■■  I  Hi  i  fl  IfW* 

I  (XXy'  ({7X— 

\  611 02F /  2  3Q4/a5  IMl 

XX)  r^ssrjpsTJ 

'Minin  I.  i'liwiw,  *  - 

UriCLASSIFl  LI) 


16.  L1’  MR  (tU  TfC.N  M-  *|T 


Approved  for  Public  lleloasc  --  Distribution  Unlimited 


1*  oi^T  Hi  uu  :  m*i  $  T  i  ••  K  n  r  .  /  iw. 


I  :  •  k  »/  .fifM"  .  *  If  ••»  ,•»  <r » 


I  U  suf  PL  I.MCNT  AbS  not  t 


19.  KEY  woncr.  t  -  fiiln 


weighted  M-estinates,  weighted  least  squares,  unknown  variances,  estimated 
weights,  asymptotically  normal 


?0.  A  f)  S 1  MAC  ,  ;  -  «•*- rl<' 


•>.f  i  '  .  r,  f-  f.v  k 


We  consider  a  heteroscedastic  linear  model  in  which  the  variances  are  a 
parametric  function  of  the  mean  responses  {i,,...,  i^'  and  a  parameter  p.  We 
propose  robust  estimates  for  the  regression  parameter  P  and  show  lint,  as  long 
as  a  reasonable  starting  estimate  of  "  is  available,  our  estimates  of  ,  are 
asymptotically  equivalent  t.o  the  natural  estimate  obtained  with  known 
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20.  variances.  A  particular  method  or  cst  into  Line)  •  is  proposed  arid  shown 
by  Monte-Carlo  to  work  quite  wc  ' ,  especially  in  power  and  exponential 
models  for  the  variances.  We  ai  >o  briefly  discuss  a  "feedback"  estimate 
of  p. 
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